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REMARKS 



Reconsideration of the application is respectfully requested for the following reasons: 



A substitute specification in proper idiomatic English is attached hereto, as required in 
item 2 on page 2 of the Official Action. 

In addition, the abstract and claims 2-4 have been amended to place them in proper 
U.S. format and to correct various minor idiomatic errors, without changing the scope of the 
claims. 

Because the changes to the specification, claims, and abstract are all formal in nature, 
it is respectfully submitted that the changes do not involve new matter. 

2. Rejection of Claim 1 Under 35 USC §1020)) 

This rejection has been rendered moot by the cancellation of claim 1. Claims 2-4 are 
allowed. 

Having thus overcome each of the rejections made in the Official Action, withdrawal 
of the rejections and expedited passage of the application to issue is requested. 



Amendments to Specification, Abstract, and Claims 



Respectfully submitted, 



BACON & THOMAS, PLLC 




By: BENJAMIN E. URCIA 
Registration No. 33,805 



Date: April 14, 2005 
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VISUAL TRACKING METHOD BY COLOR INFORMATION 

5 BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to a method of visual tracking with use of color. 

10 More particularly, the invention relates to ayisual target tracking method applicable for 
various fields in such techniques as correcting important parts of recorded images for 
broadcasting stations, tracking human feee -faces in unmanned monitoring systems, 
tracking speakers for remote conf e r e nc e conferences, or remote conversation trough 
through communication networks, setting fac e contour face contours for face recognition 

15 in security systems, tracking of a specified target, and so forth. 

The need for tracking or recognizing a target having a specific color is increased 
in various application fields like broadcasting, unmanned monitoring systems, security 
systems, remote conf e renc e trough conferencing through communication networks, 
control of unmanned flying obj e ct objects , unmanned docking systems, etc. Considering 

20 the fact that amajority of information human beings obtain is through visual means, the 
visual tracking techniques is -are expected to further expand their application fields in the 
future. 

Description of the Related Art 

25 In conventional visual tracking techniques using color information, colors of the 

target have been represented by color models[[,]] which ar e known to be robust with 
respect to illumination changes^ such as in normalized Red-Green-Blue (R-G-B) space or 
Hue-Saturation-Intensity (H-S-I) space. 

However, these color models well accommodate uniform changes of illumination, 

30 but show limitations in the-cases where the brightness of the target changes irregularly 
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due to angle changes between light source and target or the brightness changes abruptly. 

For example, when a man is walking through a gallery whose ceiling is equipped 
with fluorescent lights at uniform intervals, the relative position of the immediately 
affecting fluorescent light relative to the man keeps changing and, therefore, the 
5 brightness of the man's face keeps changing. And depending on the direction the man is 
moving, it often probable that one side of the face gets dark e n darker while the other side 
gets light e d lighter . Moreover, the surface reflectiveness of ajiuman face is hardly 
uniform due to secretion like-of sweat and it is extremely unlikely for the whole face to 
have uniform illumination changes. 
10 In other words, since color distribution changes as illumination intensity varies, 

the conventional visual tracking techniques have problems in using a color model 
normalized and set-up for uniform illumination changes. 

SUMMARY OF THE INVENTION 

15 

It is an object of the present invention to resolve the aforementioned problems by 
providing a method of visual tracking with use of color information which is robust to 
irregular or abrupt changes of illumination. 

It is another object of the present invention to provide a visual tracking method 
20 using color information in which ^characteristic of the photographing element of the 
camera is analyzed and the results of the analysis is modeled by a_B-spline curve, then 
allowing real-time visual tracking and application fer4he-to situations of rapid movement 
of the target. 

The present invention proposes a visual tracking method in which the 
25 photographing element of a_camera (CCD or CMOS) is analyzed for its -brightness 
characteristics to th e brightn e ss and the analysis results are modeled beforehand so that it 
resolves the problems of the conventional visual tracking algorithm^ which is not 
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adaptable to non-uniform or irregular change changes of illumination. 

The present invention also proposes a motion acceleration predictor for improving 
the speed of visual tracking. 

The present invention provides a visual tracking method using color information 
5 comprising: ^.three-dimensional color modeling step in which images obtained under 
various illumination conditions are analyzed and thereby the photographing 
characteristics of the camera as to the target is represented by a three-dimensional model; 
a_target recognition step in which judgement is made by the difference between the 
previous and the current images about whether or not a new target object appears, the 

10 target region is located by applying the color model in said three-dimensional color 
modeling step, and the final decision is made as to whether or not visual tracking is to be 
performed for the target depending on the shape analysis of the target region^ and the-a 
third step of visual tracking in which an arbitrary pixel is monitored and judged using 
said color model if it belongs to the target region^ and at the same time the judgement 

1 5 process has to bo being adaptable at the same time to the movement speed of the target by 
estimating the movement of the target region. 

The above and other features and advantages of the present invention will be more 
clearly understood for those skilled in the art from the following detailed description 
taken in conjunction with the accompanying drawings, which form a part of this 

20 disclosure. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a schematic block diagram representing the concept of the visual 
25 tracking method according to the present invention; 

FIG. 2 is a graph showing the average hue to the brightness value modeled by the 
present invention; 
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FIG. 3 is a graph showing the standard deviation of hue to the brightness value 
modeled by the present invention; 

FIG. 4 is a graph showing the average saturation to the brightness value modeled 
by the present invention; and 
5 FIG. 5 is a graph showing the standard deviation of saturation to the brightness 

value modeled by the present invention. 

DETAILED DESCRIPTION OF THE EMBODIMENT 

10 Hereinafter, the present invention is described in detail by referring to the 

accompanying drawings. 

First of all, the visual tracking method according to the present invention 

compris e comprises the step of three-dimensional color modeling of the target (S 1 00), the 

step of target recognition for initially perceiving the target (S200) , and the step of visual 
15 tracking for repetitive visual tracking (S300)[[;]]. and th e The basic concept of the 

invention is schematically shown in FIG. 1 . 

The first step of three-dimensional color modeling (SI 00) is the step fef-of 

establishing a three-dimensional color model, and-which has to be performed before 

initiation of the visual tracking. 
20 The above color model consists of four quadratic functions representing average 

and standard deviation of target hue with respect to brightness changes and average and 

standard deviation of target saturation. And th e s e These four quadratic functions 

constitute a single Gaussian function. 

For this purpose, the model is obtained by dividing the whole region subject to 
25 brightness changes of the image into n regions with regular sizes and by approximating 

two adjacent regions using quadratic curves, accomplishing the model for the whole 

region. 



4 



MARKED -UP COPY OF 
ORIGINAL SPECIFICATION 

Ser. No. 09/903,515 
Examiner Lavin 
Group Art Unit 2621 

Here, each region is represented by a quadratic function such as Eqn. 1 along 
with boundary conditions of the quadratic curves represented by Eqns. 2-4. In other 
words, the model is built by the region curve (Eqn. 1) along with boundary value 
condition (Eqn. 2) , continuity condition at the boundary (Eqn. 3) , and second derivative 
5 condition for the region curve (Eqn. 4) as shown in the -following: fallowings . 

[Eqn. 1] 

f j (x) =arx 2 +bfX+Ci . 

10 [Eqn. 2] 

fi (xO =arXi 2 +bfX +Ci=f (xO , for i = 1, 2, ...n-1 . 

f i (xi+0 =arxi+i 2 +brXj+i+Ci=f (x i+ i) , for i = 1, 2 , ... n-1 . 

[Eqn. 3] 

15 2-ai-Xi+i+bi s= 2-ai+i-Xi+i+bi+i, for i=l, 2, ...n-1 . 

[Eqn. 4] 
ai=0. 

20 Herein, x is the brightness of the concerning pixel, and f (x) is the average or standard 
deviation of the hue or the average or standard deviation of the saturation for the given 
brightness x. 

If the whole range of brightness is divided into 5, the relational equation for 
25 obtaining the coefficients of the quadratic curve for each region can be represented by a 
single matrix equation of Eqn. -5 as shown in the fallowings, following: 
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[Eqn. 5] 
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Herein, (a;, bj, Cj) for i=l, 2, 3, and 4 denote coefficients for a_quadratic curve 
representing each region. 

Based on four relational equations obtained above, the t hree-dimensional color 
model is represented by the following Eqns. 6 and 7; i.e., 

10 

[Eqn. 6] 

H m (i)-T h *H a (i) <H 3D (i) <H m (i)+T h *H a (i) 
[Eqn. 7] 

15 S m (i)-T h *S a (i) <S 3D (i) ^S m (0+ T h *S a (i) 

Herein, H m (/) for 0 ^ ^255 is the function of hue average, H a (z) for 0 ^ ^255 is 
the function of hue standard deviation, S m (i) for 0 <i <255 is the function of saturation 
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average, S a (i) for 0 <i <255 is the function of saturation standard deviation, and H 3D (i) 
for 0 <i <255 and S 3D (i) for 0 <i <255 are the color model functions. 

In the second step of target recognition (S200) Judgement is made about whether 
a new target object appears by the difference between the previous and the current 
5 images, the target region is located by applying the color model newly proposed in the 
previous step, and the final decision is made as to whether or not visual tracking is to be 
performed for the target by shape analysis of the target region. 

The aforementioned shape analysis is performed by comparing the actual shape of 
the image in the target region against the outlined shape inputted beforehand. For 
10 example, if ahuman face is tracked, an egg shape is chosen as the reference for contour 
comparison; if a ball is tracked, a_circular shape is chosen as ^comparison reference. 

In the third step of visual tracking (S300), the recognized target region is 
continuously tracked. 

In this step, an arbitrary pixel is monitored and judged continuously if it belongs 
1 5 to the target region using the formulated color model, while the target region movement 
is estimated and the judgement process has to be-accommodate the movement speed of 
the target. 

A Kalman filter technique might be considered for the movement prediction. 
However, it has a slow algorithmic speed due to its heavy arithmetic load^ and shows 
20 drawbacks an exhibits the drawback that prompt adaptation is impossible for abrupt 
movement because it utilizes too much of th e p ast information. 

To cope with this difficulty, the present invention employs a simple movement 
tracking 

method, which uses analysis results of the past three images as shown in Eqn. 8; i.e., 

25 

[Eqn. 8] 
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P m (0 = 




F m (i-l)-2.F m (i-2) + F m (i-3) 



At • At 



Herein, P m (i) is the predicted acceleration of the target region in the i-th image, 
F m (i) is the position of the target region in the i-th image, and At is the time increment 
between the i-th and (i-l)-th images. 



to the present invention under various illumination conditions. 

In Figs. 2 through 5, the abscissa represents brightness value while the ordinate 
represents average and standard deviation of hue and those of saturation, respectively. 
According to the present invention, the brightness of the face changes depending 

10 on the relative position of the ceiling fluorescent lamp to the face, and the tracking works 
continuously even if illumination is abruptly darkened. 

As shown previously, the present invention allows adaptable performance of a 
camera when a target with a specific color is tracked under irregular or non-uniform 
change of illumination. In other words, the camera photographing characteristics as to the 

1 5 target are three-dimensionally modeled by analyzing the images obtained under various 
brightness conditions, thus enhancing reliability against illumination changes, and a 
simple acceleration predictor with a light arithmetic burden is employed for adaptation 
fe^to movement speed change of the target. So that As a result, the present invention has 
the- an effect which can be utilized in various application fields like broadcasting, 

20 unmanned monitoring systems, security systems, remote conference trough conferencing 
through communication networks, control of unmanned flyin g obj e ct objects , unmanned 
docking systems, and so forth. 

Although the present invention has been described and illustrated in connection with the 
specific embodiments, it will be apparent for those skilled in the art that various 
25 modifications and changes may be made without departing from the idea and scope of the 
present invention set forth in the appended claims. 
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Figs. 2 through 5 show four curves which are modeled for face tracking according 



